Tiered Byzantine-Fault Tolerance for Long-Term Integrity

Long-term services that operate reliably are hard to construct. In this paper, we argue that for long-term services we need a stronger service property called Healthy-WriteImplies-Correct-Read (HWICR): once a value is written in a healthy period (i.e., when the system’s fault assumption is not violated), the value is correctly read despite intervening unhealthy periods. To build services with the HWICR property, we adapt the traditional Byzantine-fault model to a tiered fault model, that allows customizing fault assumptions to different system components; the refined fault model allows services with a long-term horizon to handle the inevitable but rare violations of more traditional fault assumptions. As a specific case study, we present TimeMachine, a Byzantine-fault tolerant keyvalue store that provides the HWICR property under a three-tier fault model. We justify the design and tiered fault model behind TimeMachine, present an implementation, and show experimental results suggesting it is a practical solution to the long-open problem of long-term archival storage integrity.